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IMPLICATIONS OF A CLASS OF 
COVARIANCE STRUCTURE MODELS FOR THE 
DEVELOPMENT OF MEASURING INSTRUMENTS 



William J. Bramble David E. Wiley 

University of Kentucky University of Chicago 



Covariance Structure Models 



Covariance structure analysis is a generic construct describing a variety of 
statistical procedures for determining sources of latent variation and covariation 
among a set of variables. The basic data for covariance structure analysis, are p- 
variate response vectorr for each of N subjects. The generic structural nodel for 
the population covariance matrix in this type of analysis is 

(1) l - A * A 1 + ' ¥ 

pxp pxm mxm mxp pxp 

In equation (I) l is the pxp population covariance matrix, A is a matrix of weights 
relating the observed p variables to a set of m latent variables, is the mxm co- 
variance matrix for the latent variables, and f is a diagonal matrix of error variances 
for the p observed variables. A set of covariance structural models is generated when 
alternative restrictions are placed on the parameter matrices of the general model. 
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Wiley (1967) reviewed the work of Bock and Bargmann (1966) , Scheffe (1956) , Lawley 
(1940, 1942), and Joreskog (1967) and delineated a sat of sixteen possible covariance 
structure models based on the following restrictions on the parameter matrices of the 
general nodel: 
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Restrictions of the matrix of weights on the latent variables (A); 

(1) A may be completely unspecified (all elements of A 
to be estimated) . 

(2) A may contain some specified elements and have othrr 
elements which must be estimated (represented by A*). 
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(3) A may be completely specified, but scaled by some unknown but 
estimable matrix of scaling weights (represented by TA) . 

(4) A may be completely specified and unsealed (represented by A). 

(b) Restrictions on the covariance matrix for the latent variables ($) . 

(1) The latent variables are assumed uncorrelated (orthogonal case) 
and thus $ is diagonal (4> * A 2 )* 

(2) The latent variables are assumed correlated (oblique case) and 
thus 4 is not restricted to a diagonal matrix (thus we merely 
use $ in this case) . 

(c) Restrictions on the matrix of error variances. 

(1) The p error variances are assumed homogeneous (Y * Io 2 ). 

(2) The error variances are assumed heterogeneous (¥ general diagonal). 

in t he above restrictions those on A depend on the theoretical preconceptions of 
the experimenter i the situation, and the form of the data. The general A matrix in 
which all elements are estimated is typically employed in classical factor analysis 
(see Harman, 1967) and in maximum likelihood factor analysis (see Lawley, 1940, 1942). 
Including the restricted case of A (i.e., A*) provides a variant of these procedures. 

In the case of A completely specified (A • A) the usual form of A is that of a repa- 
ramaterized mixed model analysis of variance design matrix where some m-factor design 
is employed and the p variables are the p treatment combinations of the m-factor 
design. If there is a problem with varying metric across the p treatment combinations 
the design matrix may be rescaled (A • TA) . The present paper involves models in 
which the latter two restrictions on A are employed (A * A and A * TA) . The faetrlc- 
tions Employed on the 6thet two parameter matrices (♦ and f)have to do with assump- 
tions that are made about these parameter matrices rather than preconceptions of the 







2 



- 3 - 



experimenter . In addition we want to have the opportunity to test the validity of 
these assumptions in empirical situations. The procedures presented in this paper 
provide us this opportunity. 

The class of covariance structure models described here is the 2x2x2 array 
of eight models presented in Table 1. 



TABLE 1 

THE CLASS OF COVARIANCE STRUCTURE MODELS 



Restrictions on A 



Y-IO 2 

Restrictions 
on Y 



A 


TA 


Restrictions on $ 

A 2 4 A 2 * 


E^A 2 A'+I 0 2 


E«A*A’+Io 2 


3>rAA 2 A'r+la 2 


£-rA*A'r+io 2 


z«aa 2 a'+y 


Z-A^A'+Y 


E«rAA 2 A'm 


E-rAGAT+Y 



Notice in Table 1 that the models range in degree of restriction from l * AA 2 A* + I o 2 
(most restricted) to l * TAtA’r + ¥ (least restricted). 

The data for the set of eight covariance structure models is like that depicted 
in Figure 1. It is obtained from the p repeated measures of an analysis of variance 
design for each of n subjects. If we assume that the observation vectors (y^) are 
randomly drawn from a multivariate normal population with mean vector (p) and popula- 
tion covariance matrix £ we can apply maximum likelihood procedures in estimating the 
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Figure 1. Data for a repeated measures analysis of variance design. 

elements of the parameter matrices of the structural model for The log likelihood 
Is given as 

(2) Log L - ^ log 2n - ^ log | £ | - y tr F~' S, 

where S Is the sample covariance matrix. The general expression for the first deriva- 
tives of the likelihood functions with respect to a general element x ±& 

(3) « *2~ tr {|$ fa" 1 S E -1 - F" 1 )}. 

Using this expression the first derivatives of the log likelihood function with respect 
to each of the parameter matrices of the model have been obtained by Bramble, Schmidt, 

and Wiley (1969, 1970). 
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An efficient numerical method for solving the likelihood equations was found by 
these authors in the quasi-Newton method originated by Davidon (1959) and modified by 
Fletcher and Powell (1963) . A FORTRAN IV program has been developed which implements 
these procedures obtaining point estimates for the elements of the parameter matrices 
of the structural model* 

An estimate of the asymptotic variance ** covariance matrix of the estimates is 
provided from the negative of the inverse of the final estimate of the matrix of second 
derivatives or its expected value. The computer program is written using the matrix 
subroutine package of Bock and Peterson (1967) and is available from either of the 
authors* 



Testing the Fit of Covariance Structure Models 

The computer program also provides a chi square approximation of the likelihood 
ratio test of fit of a particular model 

(4) x 2 - -2 In X 

* -N In |i| 

whefe the degrees of freedom are equal to the difference in the number of parameters 
estimated between the alternative and null models. Model testing can also be carried 
out sequentially, that Is, the relative fit of alternative models can be tested using 
the likelihood ratio statistic 

(5) X - jji+l| l/2N . 

TCP 72 " 

In equation (5) the estimated covariance matrix for the less restricted of the two Models 
being considered is denoted by and that for the more restricted model by fl. The 

chi square approximation for this statistic la 
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(6) X 2 r , Nln ■ X 2 i - X 2 i +1 

i^'i+L 

Thus the relative fit of any two models can be tested by the difference between the x 2 
approximations for the likelihood ratio tests of fit for the two models. Sequential 
model testing of this type has been developed by Bramble (1970) and is useful in many 
applications of covariance structure analysis including applications to measurement 
problems . 
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APPLICATION OF THE COVARIANCE STRUCTURE M3DEL3 TO 



MEASUREMENT PROBLEMS 



Using the procedures outlined in the foregoing sections we can develop measuring 
instruments in which we know the relationships among traits and item characteristics 
more precisely than has heretofore been possible. Using these procedures the item to 
be Included in the test must be written or selected in such a way as to very exactly 
represent within the test the types of variance and covariance which are of signifi- 
cant interest to merit study. Thus, we do not begin with a loose conglomerate of un- 
selected items, but one in which properties of interest are built into the test 
systematically. This procedure in and of itself is an apparent improvement in test 
development techniques, but the information made available from a covariance structure 
analysis of an instrument designed in this way is of considerably more value. 

Not only can we obtain an estimate of the relative amounts of variance in the 
latent tralti represented in a set of items, but we get an estimate of the covariance 
between these latent traits (not merely between the observed traits) and have the 
opportunity to test the hypothesis that the covariances between latent traits are 
zero. We do this by testing the assumptions about the $ matrix in the structural 
model (i.e., by looking at whether this matrix should be oblique or orthogonal). The 
relative sizes of the diagonal elements of ♦ tell us the importance of (i.e., the 
variance contributed by) each of the latent traits. Information about the latent trait9 
can guide us in the future development of the instrument. The form of Y yields homogenei 
information about the error variances. 

1 Characteristics of a test which can be studied in this way include the number 

importance of the latent variables included in the test and the relationship aaong 
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these variables. We may also study characteristics of the items such as phrasing, 
direction of wording, etc. and we can look at the relation of these characteristics, 
treated as latent variables, tc the substantive traits which are represented. 

To give a simple example of an application of the procedures to a measurement 
problem consider the case of constructing a test battery to measure two types of latent 
ability (e.g., computational ability and logical reasoning). A test constructor may 
be aware of several methods which have been tried out to measure each type of ability. 
Let us say that two measures of computational ability (C^ and C2) and two of logical 
reasoning (R^ and R2) seem particularly appropriate for a given application. Thus a 
teat might be designed which contained four subtests of homogeneous items which measure 
each of the four traits (C^, C2> R^t and R2) . 

The matrix of weights (A) for the struetuial model is 



C R 

~T 0 

1 0 

(7) A * 01 

0 1 



Alternatively, the matrix A could have taken the form of a reparametorlzed analysis of 
variance design matrix. An A matrix of this type would include a column for the grand 
mean and a column for the contrasts of the fixed effect part of the analysis of vari- 
ance design. If we administered the items to a group of subjects and computed the 
sample covariance matrix we could then obtain maximum likelihood estimates for the 
elements of particular structural models. We could test the assumption that the latent 
variables (C and R) are uncorrelated by testing the assumption that a model with * 
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orthogonal fits the data. We could test the assumption that the subtests have equal 
variances by testing the assumption that Y is homogeneous. 



Consider, for ease of explanation, the case where only the restriction ^ on A is 
employed (i.e., four models are run) and where the most general model fits and more 
restricted models do not. The parameters to be estimated are those in 
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The relative amount of variance for the latent variables can be assessed by looking at 
the diagonal elements of <J. The degree of relationship between the two latent variables 
is assessed by looking at the off-diagonal element in #. The error components can 

be compared fcjr looking at the diagonal elements of Y. 



To expand the problem somewhat, the investigator might also be interested in some 

item characteristics, e.g. , direction of wording D and strength of the item S, and their 

relationship to the latent variables. Strength refers in this case to the level at 

which a positive response to an item manifests a particular response (e.g. , agreeing 

with very positively phrased items may represent a stronger response than agreeing 

2 

with a more mildly stated form of the same item) . A 2 factorial design for the 
characteristics of the items crossed with the original design yields 
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C vs . R 


D 


S 


°in 


+1 


+1 


+1 


+1 


C 112 


+1 


+1 


+1 


-1 


< 9) A ’ C 121 


+1 


+1 


-1 


+1 


CM 

CM 


+1 


+1 


-1 


-1 


c 2n 


+1 


+1 


+1 


+1 


c 2l2 


+1 


+1 


+1 


-1 


c 221 


+1 


+1 


-1 


+1 


c 222 


+1 


+1 


-1 


-1 


R lll 


+1 


-1 


+1 


+1 


R ll2 


+1 


-I 


+1 


-1 


R l21 


+1 


• 1 


•1 


-*-1 


r 122 


+1 


-1 


-1 


-1 


*211 


+1 


-1 


+1 


+1 


R 212 


+1 


-1 


+1 


-1 


R 221 


+1 


-*1 


-1 


+1 


R 222 


+1 


-1 


-1 


-1 



Analyzing the model containing the above A matrix, a 4x4 matrix and a 16 x 16 
¥ matrix we can investigate the structure of the latent trait 9 having to do with ability, 
those having to do with item characteristics, as well as the interrelationships between 
the two type9 of traits. In thi9 way more precise measuring instruments can be developed. 
One question we can answer has to do with the positively and negatively phrased item 9 . 

Does the reflection of an item mean that we merely reflect the scoring of an item or 
alternative form9 of response be weighted differently? Looking at the covariance 

0 
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between the C vs R contrast and D sheds light on this issue. Similar questions 
can be asked about item strength and investigated in an analogous manner. 
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NUMERICAL EXAMPLE 

An investigation of content acquiescence in the MMPI has been reported by Bock, 
Dicken, and Van Pelt (1969). These authors studied response acquiescence in two scales 
(Pt and Hy) most representative of the two orthogonal factors which repeatedly occur 
in factor analyses of the MMPI. The 108 non-overlapping items of the Pt and Hy scales 
were administered to a group of 81 undergraduates from San Diego State College in 
both their original and reversed forms. Item reversals were taken from the reversal 
of the complete MMPI by Dicken and Van Pelt (1967). The 216 item test was administered 
twice to the students, a week ^oart, and the results analyzed separately by MMPI scale. 
Thus S and £ are 4x4 covariance matrices. 

For the two directions of wording within an MMPI scale the response models are 

(10) Xit - P K + + a lt + 6it + eit. 

and 

(11) y lt - Py + Yi ~ a lt + <$it + C lt , 

where p x and Py are the fixed means for the scales, y^ -^s the content component for 

subject i, aj is the acquiescence component for subject i, is the component on 
occasion t due to trait Instability for subject 1, and e^ t and are the components 
within - form response error for the two scales. This is a typical mixed model anova 
and the usual restrictions on y, a, and 6 apply. The A matrix is 

1/2 1/2 1/2 

1/2 1/2 - 1/2 

1/2 - 1/2 1/2 

1/2 - 1/2 - 1/2 



(12) A - 
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and ♦ i6 given as 
(13) 

$ « 




symmetric 
°u6 ' 



The sample covariance matrices reported by Bock, Dicken, and Van Pelt for the two 
scales are given in tables 2 and 3. 



TABLE 2 

Pt Sample Covariance Matrix 



Form 


Administration 


Original 


Reversed 






1 


2 


3 4 


Original 


1 

2 


34.819 

30.319 34.997 


symmetric 


Reversed 


1 


28.178 


24.844 


29.376 




2 


25.519 


27.612 


22.323 26.780 



TABLE 3 

Hy Sample Covariance Matrix 
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A covariance structure analysis was performed on the two covariance matrices given 
in tables 2 and 3 using the four covariance structure models that contain an unsealed A 
matrix* The chi square testa of fit for the individual models are reported in table 4. 



TABLE 4 

Chi Square Tests of Pit for the Pt and Hy Data 



$ Orthogonal 



$ Oblique 



In table 4 it Is seen that the most restricted model fits for Hy, but not for Pt. The 
model containing • oblique and Y ■ I o 2 fits for Pt and the difference chi square con- 
trasting this model with the most restricted model is 11.99 (d.f. -3). The latter value 
Is significant beyond the .01 level. Thus, the oblique case of ♦ is necessary for the 
Pt data, but not for the Hy data. 

The estimates for the elements of the parameter matrices are given in table 5. 

The estimates given for Pt and Hy are those obtained by using the appropriate model for 
each. 





P 


* a^I 






general 






Pt 


14.63 


d.f.- 6 


p<.025 


12.70 


d.f. -3 


p<*01 




Hy 


9.35 


d.f.- 26 


p<.20 


6.04 


d.f. -3 


p< . 15 




pt 


2.64 


d.f .-3 


p<*55 


.72 


— 


— 




HV 


3.77 


d.f.- 3 


p<. 35 


.07 


— 


— 
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TABLE 5 







Parameter 


Estimates 


for 


the Pt and Hy Data 






Raw Estimates* 


S.E. 




Full 


Estimate 


Parameter 


pt 


HU 


Pt 


HZ 


Parameter 


pt 




hi 


10.42 


6.02 


.72 


.50 


hi 


108.56 


36.20 


hi 


.72 


0.00 


.21 


— 


*21 


7.52 


0.00 


*22 


1.40 


6.02 


.25 


.26 


*22 


2.48 


2.47 


*31 


.08 


0.00 


.26 




*31 


.88 


0.00 


*32 


-.78 


0.00 


.42 


— 


*32 


-1.03 


0.00 


*33 


2.19 


4.90 


.28 


• 27 


*33 


5.43 


6.00 


a . 


1.54 


1.52 


.11 


.12 


ff2 « 


2.36 


2.32 



^ The raw estimates include the elements of the Cholesky factorisation of 4 (which 
are represented by t^j) and the square root of the error variance ♦ 

from the full parameter estimates we can see that the content variance for both teats 
was quite large (108.56 for Pt and 36.20 for Hy) relative to the acquiescence variance 
(2.48 for Pt and 2.47 for Hy) end variance due to trait instability (5.43 for Pt and 
6.00 for tty). The error variances for the two tests ware relatively consistent (2.36 
for Pt and 2*32 for Hy). Tha interesting off-diagonal term in 4 is 4^ In Pt (7.52). 
In this case even though the variance due to acquiescence is relatively small the con- 
tent by acquiescence covariance is rather large. 
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